Skip to content

introduce user groups#1621

Open
t-lenz wants to merge 12 commits intocms-dev:mainfrom
ioi-germany:groups
Open

introduce user groups#1621
t-lenz wants to merge 12 commits intocms-dev:mainfrom
ioi-germany:groups

Conversation

@t-lenz
Copy link

@t-lenz t-lenz commented Jan 18, 2026

We've been using CMS for the German IOI selection for many years now, and we often have offsite contestants who cannot compete at the same time as the onsite contestants, e.g. because they are ill at the time of the contest or because they live in a different timezone.

If one wants to allot different time slots for the users of a contest in vanilla CMS, this would require setting delay (and possibly extratime) for users individually. This can get pretty inconvenient, in particular if there are multiple contestants affected. Moreover, this does not allow having one group of contestants (in the above example, the onsite contestants) compete in a fixed timeslot and others in a timeslot of their choice (USACO style).

This pull request changes the DB format to introduce user groups. Participations are now assigned a group, and start, stop, per_user_time, etc. are no longer properties of a contest, but instead of a user group. In the above example usecase, one could then simply have one group for the onsite contestants and one for offsite contestants, and one could e.g. have the first group compete at a fixed time, with the offsite group being able to (more or less) freely choose a timeslot. As a proof of concept, we have also adjusted the Italian yaml loader so that it is able to assign users to groups; old yaml configs are still valid and result in all users being assigned to the same group.

This is based on code that has been in use in the German fork of CMS for over 10 years now, but has been cleaned up and updated for the latest version of CMS. Most of the original code was written by @fagu and @magula; the present version also contains contributions by @chuyang-wang-dev.

@t-lenz
introduce user groups
ef2d6be 
much of this is based on code originally written by @fagu and @magula
prandla
prandla requested changes Feb 11, 2026
View reviewed changes
Copy link
Member

@prandla prandla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi, sorry for taking this long to get around to this!

overall, we (or well, at least Luca and i) agree that this would be a good feature to have. i left some comments, some of them minor, some of them requiring more work. i could do all of these fixes myself too, though then i think someone else will need to re-review it :)

in addition to the inline comments, we will need to fix the test suite failures (and possibly add new tests, but currently we don't have a good way to write tests for AWS UI, so i think it's fine to skip that for now). also, we need a DumpUpdater and an sql migration. also, you should run Ruff on modified lines, and some places could use more type hints.

and please do let me know if you disagree with some of the proposed changes, i'm always up for some healthy debate :)

j-bmn added 3 commits February 14, 2026 16:39
@j-bmn
Fix missing name for implicitly-created main group
84b8ba3
@j-bmn
Gender-neutral help text for participant's group
383aac6
@j-bmn
Fix (most) failing tests
b1ee39d 
Should fix all failures except DumpImporterTest and schema_diff_test
(which both likely need more work).
@codecov
Copy link

codecov bot commented Feb 14, 2026 •
edited
Loading

❌ 6 Tests Failed:

Tests completed Failed Passed Skipped
695 6 689 8
View the top 3 failed test(s) by shortest run time 
cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_skip_generated
Stack Traces | 0.021s run time 
self = <sqlalchemy.engine.base.Connection object at 0x7f75ef56bf90>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75ef41b3d0>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef56a650>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506890; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef56a650>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (4, null, null, 00:00:00, 00:00:00, null, f, f, 8, 4, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_skip_generated>

    def test_import_skip_generated(self):
        """Test importing everything but the generated data."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import(skip_generated=True))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:261: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506890; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 8, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef56a650>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (4, null, null, 00:00:00, 00:00:00, null, f, f, 8, 4, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 8, 'user_id': 4, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError
cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_skip_files
Stack Traces | 0.022s run time 
self = <sqlalchemy.engine.base.Connection object at 0x7f75ef62fc50>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75f07ef250>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f07ef5d0>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5896c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f07ef5d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (3, null, null, 00:00:00, 00:00:00, null, f, f, 6, 3, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_skip_files>

    def test_import_skip_files(self):
        """Test importing the json but not the files."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import(load_files=False))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:277: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5896c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 6, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f07ef5d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (3, null, null, 00:00:00, 00:00:00, null, f, f, 6, 3, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 6, 'user_id': 3, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError
cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_old
Stack Traces | 0.057s run time 
self = <sqlalchemy.engine.base.Connection object at 0x7f75ef4f0bd0>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75ef4fe850>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef4fc610>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5064d0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef4fc610>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (2, null, null, 00:00:00, 00:00:00, null, f, f, 4, 2, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_old>

    def test_import_old(self):
        """Test importing an old dump.
    
        This does not pretend to be exhaustive, just makes sure the happy
        path of the updaters run successfully.
    
        """
        self.write_dump({
            "contest_key": {
                "_class": "Contest",
                "name": "contestname",
                "description": "contest description",
                "start": 1_234_567_890.000,
                "stop": 1_324_567_890.000,
                "token_initial": 2,
                "token_gen_number": 1,
                "token_gen_time": 10,
                "token_total": 100,
                "token_max": 100,
                "tasks": ["task_key"],
            },
            "task_key": {
                "_class": "Task",
                "name": "taskname",
                "title": "task title",
                "num": 0,
                "primary_statements": "[\"en\", \"ja\"]",
                "token_initial": None,
                "token_gen_number": 0,
                "token_gen_time": 0,
                "token_total": None,
                "token_max": None,
                "task_type": "Batch",
                "task_type_parameters": "[]",
                "score_type": "Sum",
                "score_type_parameters": "[]",
                "time_limit": 0.0,
                "memory_limit": None,
                "contest": "contest_key",
                "attachments": {},
                "managers": {},
                "testcases": {},
                "submissions": ["sub1_key", "sub2_key"],
                "user_tests": [],
            },
            "user_key": {
                "_class": "User",
                "username": "username",
                "first_name": "First Name",
                "last_name": "Last Name",
                "password": "pwd",
                "email": "",
                "ip": "0.0.0.0",
                "preferred_languages": "[\"en\", \"it_IT\"]",
                "contest": "contest_key",
                "submissions": ["sub1_key", "sub2_key"],
            },
            "sub1_key": {
                "_class": "Submission",
                "timestamp": 1_234_567_890.123,
                "language": "c",
                "user": "user_key",
                "task": "task_key",
                "compilation_text": "OK [1.234 - 20]",
                "files": {},
                "executables": {"exe": "exe_key"},
                "evaluations": [],
            },
            "sub2_key": {
                "_class": "Submission",
                "timestamp": 1_234_567_900.123,
                "language": "c",
                "user": "user_key",
                "task": "task_key",
                "compilation_text": "Killed with signal 11 [0.123 - 10]\n",
                "files": {},
                "executables": {},
                "evaluations": [],
            },
            "exe_key": {
                "_class": "Executable",
                "submission": "sub1_key",
                "filename": "exe",
                "digest": TestDumpImporter.GENERATED_FILE_DIGEST,
            },
            "_version": 1,
            "_objects": ["contest_key", "user_key"],
        })
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import(skip_generated=True))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:394: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef5064d0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 4, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef4fc610>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (2, null, null, 00:00:00, 00:00:00, null, f, f, 4, 2, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 4, 'user_id': 2, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError
cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import
Stack Traces | 0.262s run time 
self = <sqlalchemy.engine.base.Connection object at 0x7f75f07d0990>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75f08d3290>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef6581d0>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75f397e5c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef6581d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 2, 1, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import>

    def test_import(self):
        """Test importing everything, while keeping the existing contest."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
>       self.assertTrue(self.do_import())
                        ^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:224: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75f397e5c0; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 2, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75ef6581d0>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 2, 1, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 2, 'user_id': 1, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError
cmstestsuite/unit_tests/cmscontrib/DumpImporterTest.py::TestDumpImporter::test_import_with_drop
Stack Traces | 0.321s run time 
self = <sqlalchemy.engine.base.Connection object at 0x7f75ef51b110>
dialect = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2'>>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
args = (<sqlalchemy.dialects.postgresql.psycopg2.PGCompiler_psycopg2 object at 0x7f75f07a30d0>, [{'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}])
conn = <sqlalchemy.pool.base._ConnectionFairy object at 0x7f75ef518dd0>
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f17b0a50>

    def _execute_context(
        self, dialect, constructor, statement, parameters, *args
    ):
        """Create an :class:`.ExecutionContext` and execute, returning
        a :class:`_engine.ResultProxy`.
    
        """
    
        try:
            try:
                conn = self.__connection
            except AttributeError:
                # escape "except AttributeError" before revalidating
                # to prevent misleading stacktraces in Py3K
                conn = None
            if conn is None:
                conn = self._revalidate_connection()
    
            context = constructor(dialect, self, conn, *args)
        except BaseException as e:
            self._handle_dbapi_exception(
                e, util.text_type(statement), parameters, None, None
            )
    
        if context.compiled:
            context.pre_exec()
    
        cursor, statement, parameters = (
            context.cursor,
            context.statement,
            context.parameters,
        )
    
        if not context.executemany:
            parameters = parameters[0]
    
        if self._has_events or self.engine._has_events:
            for fn in self.dispatch.before_cursor_execute:
                statement, parameters = fn(
                    self,
                    cursor,
                    statement,
                    parameters,
                    context,
                    context.executemany,
                )
    
        if self._echo:
            self.engine.logger.info(statement)
            if not self.engine.hide_parameters:
                self.engine.logger.info(
                    "%r",
                    sql_util._repr_params(
                        parameters, batches=10, ismulti=context.executemany
                    ),
                )
            else:
                self.engine.logger.info(
                    "[SQL parameters hidden due to hide_parameters=True]"
                )
    
        evt_handled = False
        try:
            if context.executemany:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_executemany:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_executemany(
                        cursor, statement, parameters, context
                    )
            elif not parameters and context.no_parameters:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute_no_params:
                        if fn(cursor, statement, context):
                            evt_handled = True
                            break
                if not evt_handled:
                    self.dialect.do_execute_no_params(
                        cursor, statement, context
                    )
            else:
                if self.dialect._has_events:
                    for fn in self.dialect.dispatch.do_execute:
                        if fn(cursor, statement, parameters, context):
                            evt_handled = True
                            break
                if not evt_handled:
>                   self.dialect.do_execute(
                        cursor, statement, parameters, context
                    )

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506e30; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f17b0a50>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       psycopg2.errors.NotNullViolation: null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 1, 1, null, null).

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: NotNullViolation

The above exception was the direct cause of the following exception:

self = <DumpImporterTest.TestDumpImporter testMethod=test_import_with_drop>

    def test_import_with_drop(self):
        """Test importing everything, but dropping existing data."""
        self.write_dump(TestDumpImporter.DUMP)
        self.write_files(TestDumpImporter.FILES)
    
        # Need to close the session and reopen it, otherwise the drop hangs.
        self.session.close()
>       self.assertTrue(self.do_import(drop=True))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^

.../unit_tests/cmscontrib/DumpImporterTest.py:244: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../unit_tests/cmscontrib/DumpImporterTest.py:150: in do_import
    skip_users=skip_users).do_import()
                           ^^^^^^^^^^^
cmscontrib/DumpImporter.py:312: in do_import
    session.flush()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2540: in flush
    self._flush(objects)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2681: in _flush
    with util.safe_reraise():
.........................................................../cms/lib/python3.11.../sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/session.py:2642: in _flush
    flush_context.execute()
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:422: in execute
    rec.execute(self)
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/unitofwork.py:586: in execute
    persistence.save_obj(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:239: in save_obj
    _emit_insert_statements(
.........................................................../cms/lib/python3.11.../sqlalchemy/orm/persistence.py:1135: in _emit_insert_statements
    result = cached_connections[connection].execute(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1011: in execute
    return meth(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/sql/elements.py:298: in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1124: in _execute_clauseelement
    ret = self._execute_context(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1316: in _execute_context
    self._handle_dbapi_exception(
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1510: in _handle_dbapi_exception
    util.raise_(
.........................................................../cms/lib/python3.11.../sqlalchemy/util/compat.py:182: in raise_
    raise exception
.........................................................../cms/lib/python3.11.../sqlalchemy/engine/base.py:1276: in _execute_context
    self.dialect.do_execute(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f75f2c9dc90>
cursor = <cursor object at 0x7f75ef506e30; closed: -1>
statement = 'INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, us...d)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id'
parameters = {'contest_id': 1, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'group_id': None, ...}
context = <sqlalchemy.dialects.postgresql.psycopg2.PGExecutionContext_psycopg2 object at 0x7f75f17b0a50>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "group_id" of relation "participations" violates not-null constraint
E       DETAIL:  Failing row contains (1, null, null, 00:00:00, 00:00:00, null, f, f, 1, 1, null, null).
E       
E       [SQL: INSERT INTO participations (ip, starting_time, delay_time, extra_time, password, hidden, unrestricted, contest_id, user_id, group_id, team_id) VALUES (CAST(%(ip)s AS CIDR[])::CIDR[], %(starting_time)s, %(delay_time)s, %(extra_time)s, %(password)s, %(hidden)s, %(unrestricted)s, %(contest_id)s, %(user_id)s, %(group_id)s, %(team_id)s) RETURNING participations.id]
E       [parameters: {'ip': None, 'starting_time': None, 'delay_time': datetime.timedelta(0), 'extra_time': datetime.timedelta(0), 'password': None, 'hidden': False, 'unrestricted': False, 'contest_id': 1, 'user_id': 1, 'group_id': None, 'team_id': None}]
E       (Background on this error at: http://sqlalche..../e/13/gkpj)

.........................................................../cms/lib/python3.11.../sqlalchemy/engine/default.py:608: IntegrityError
cmstestsuite/unit_tests/schema_diff_test.py::TestSchemaDiff::test_schema_diff
Stack Traces | 0.636s run time 
self = <cmstestsuite.unit_tests.schema_diff_test.TestSchemaDiff testMethod=test_schema_diff>

    def test_schema_diff(self):
        dirname = os.path.dirname(__file__)
        schema_file = os.path.join(dirname, "schema_v1.5.sql")
        updater_file = os.path.join(dirname, "../...../cmscontrib/updaters/update_from_1.5.sql")
        updated_schema = split_schema(get_updated_schema(schema_file, updater_file))
        fresh_schema = split_schema(get_fresh_schema())
        errors = compare_schemas(updated_schema, fresh_schema)
        self.longMessage = False
>       self.assertTrue(errors == "", errors)
E       AssertionError: Statement differs between updated and fresh schema:
E       CREATE TABLE public.contests (
E       -     CONSTRAINT contests_check CHECK ((start <= stop)),
E       -     CONSTRAINT contests_check1 CHECK ((stop <= analysis_start)),
E       -     CONSTRAINT contests_check2 CHECK ((analysis_start <= analysis_stop)),
E       -     CONSTRAINT contests_check3 CHECK ((token_gen_initial <= token_gen_max)),
E       ?                              -
E       +     CONSTRAINT contests_check CHECK ((token_gen_initial <= token_gen_max)),
E             CONSTRAINT contests_max_submission_number_check CHECK ((max_submission_number > 0)),
E             CONSTRAINT contests_max_user_test_number_check CHECK ((max_user_test_number > 0)),
E             CONSTRAINT contests_min_submission_interval_check CHECK ((min_submission_interval > '00:00:00'::interval)),
E             CONSTRAINT contests_min_submission_interval_grace_period_check CHECK ((min_submission_interval_grace_period > '00:00:00'::interval)),
E             CONSTRAINT contests_min_user_test_interval_check CHECK ((min_user_test_interval > '00:00:00'::interval)),
E             CONSTRAINT contests_per_user_time_check CHECK ((per_user_time >= '00:00:00'::interval)),
E             CONSTRAINT contests_score_precision_check CHECK ((score_precision >= 0)),
E             CONSTRAINT contests_token_gen_initial_check CHECK ((token_gen_initial >= 0)),
E             CONSTRAINT contests_token_gen_interval_check CHECK ((token_gen_interval > '00:00:00'::interval)),
E             CONSTRAINT contests_token_gen_max_check CHECK ((token_gen_max > 0)),
E             CONSTRAINT contests_token_gen_number_check CHECK ((token_gen_number >= 0)),
E             CONSTRAINT contests_token_max_number_check CHECK ((token_max_number > 0)),
E             CONSTRAINT contests_token_min_interval_check CHECK ((token_min_interval >= '00:00:00'::interval)),
E             allow_password_authentication boolean NOT NULL,
E             allow_questions boolean NOT NULL,
E             allow_registration boolean NOT NULL,
E             allow_unofficial_submission_before_analysis_mode boolean NOT NULL,
E             allow_user_tests boolean NOT NULL,
E             allowed_localizations character varying[] NOT NULL,
E       -     analysis_enabled boolean NOT NULL,
E       -     analysis_start timestamp without time zone NOT NULL,
E       -     analysis_stop timestamp without time zone NOT NULL,
E             block_hidden_participations boolean NOT NULL,
E             description character varying NOT NULL,
E             id integer NOT NULL,
E             ip_autologin boolean NOT NULL,
E             ip_restriction boolean NOT NULL,
E             languages character varying[] NOT NULL,
E       +     main_group_id integer,
E             max_submission_number integer,
E             max_user_test_number integer,
E             min_submission_interval interval,
E             min_submission_interval_grace_period interval,
E             min_user_test_interval interval,
E             name public.codename NOT NULL,
E             per_user_time interval,
E             score_precision integer NOT NULL,
E       -     start timestamp without time zone NOT NULL,
E       -     stop timestamp without time zone NOT NULL,
E             submissions_download_allowed boolean NOT NULL,
E             timezone character varying,
E             token_gen_initial integer NOT NULL,
E             token_gen_interval interval NOT NULL,
E             token_gen_max integer,
E             token_gen_number integer NOT NULL,
E             token_max_number integer,
E             token_min_interval interval NOT NULL,
E             token_mode public.token_mode NOT NULL,
E         );
E       Statement differs between updated and fresh schema:
E       CREATE TABLE public.participations (
E             CONSTRAINT participations_delay_time_check CHECK ((delay_time >= '00:00:00'::interval)),
E             CONSTRAINT participations_extra_time_check CHECK ((extra_time >= '00:00:00'::interval)),
E             contest_id integer NOT NULL,
E             delay_time interval NOT NULL,
E             extra_time interval NOT NULL,
E       +     group_id integer NOT NULL,
E             hidden boolean NOT NULL,
E             id integer NOT NULL,
E             ip cidr[],
E             password character varying,
E             starting_time timestamp without time zone,
E             team_id integer,
E             unrestricted boolean NOT NULL,
E             user_id integer NOT NULL,
E         );
E       Fresh schema contains extra statement:
E       CREATE TABLE public.groups (
E           CONSTRAINT groups_check CHECK ((start <= stop)),
E           CONSTRAINT groups_check1 CHECK ((stop <= analysis_start)),
E           CONSTRAINT groups_check2 CHECK ((analysis_start <= analysis_stop)),
E           CONSTRAINT groups_per_user_time_check CHECK ((per_user_time >= '00:00:00'::interval)),
E           analysis_enabled boolean NOT NULL,
E           analysis_start timestamp without time zone NOT NULL,
E           analysis_stop timestamp without time zone NOT NULL,
E           contest_id integer,
E           id integer NOT NULL,
E           name character varying NOT NULL,
E           per_user_time interval,
E           start timestamp without time zone NOT NULL,
E           stop timestamp without time zone NOT NULL,
E       );
E       Fresh schema contains extra statement:
E       ALTER TABLE public.groups OWNER TO postgres;
E       Fresh schema contains extra statement:
E       CREATE SEQUENCE public.groups_id_seq
E           AS integer
E           START WITH 1
E           INCREMENT BY 1
E           NO MINVALUE
E           NO MAXVALUE
E           CACHE 1;
E       Fresh schema contains extra statement:
E       ALTER TABLE public.groups_id_seq OWNER TO postgres;
E       Fresh schema contains extra statement:
E       ALTER SEQUENCE public.groups_id_seq OWNED BY public.groups.id;
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ALTER COLUMN id SET DEFAULT nextval('public.groups_id_seq'::regclass);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_contest_id_name_key
E       UNIQUE (contest_id, name);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_id_contest_id_key
E       UNIQUE (id, contest_id);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_pkey
E       PRIMARY KEY (id);
E       Fresh schema contains extra statement:
E       CREATE INDEX ix_contests_main_group_id ON public.contests USING btree (main_group_id);
E       Fresh schema contains extra statement:
E       CREATE INDEX ix_groups_contest_id ON public.groups USING btree (contest_id);
E       Fresh schema contains extra statement:
E       CREATE INDEX ix_participations_group_id ON public.participations USING btree (group_id);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.contests ADD CONSTRAINT fk_contest_main_group_id
E       FOREIGN KEY (main_group_id) REFERENCES public.groups(id) ON UPDATE CASCADE ON DELETE SET NULL;
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.groups ADD CONSTRAINT groups_contest_id_fkey
E       FOREIGN KEY (contest_id) REFERENCES public.contests(id) ON UPDATE CASCADE ON DELETE CASCADE;
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.participations ADD CONSTRAINT participations_group_id_contest_id_fkey
E       FOREIGN KEY (group_id, contest_id) REFERENCES public.groups(id, contest_id);
E       Fresh schema contains extra statement:
E       ALTER TABLE ONLY public.participations ADD CONSTRAINT participations_group_id_fkey
E       FOREIGN KEY (group_id) REFERENCES public.groups(id) ON UPDATE CASCADE ON DELETE CASCADE;

cmstestsuite/unit_tests/schema_diff_test.py:173: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

prandla
prandla reviewed Feb 15, 2026
View reviewed changes
prandla
prandla requested changes Feb 15, 2026
View reviewed changes
Copy link
Member

@prandla prandla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noticed a few more things when reading through the code one more time, and poking around in AWS a bit. i think after addressing these comments, and the foreign key constraint mentioned in the previous discussion, the code itself is fine. again we still need a DumpUpdater and a sql updater. i can write those myself if you'd prefer (i assume you haven't needed to deal with those features in your fork...)

@t-lenz
Copy link
Author

t-lenz commented Feb 16, 2026

This should address the requested changes (I also fixed a typo in the fallback page for AddGroupHandler). It would indeed be great if you could take care of the DumpUpdater and SQL updater. Thanks a lot!

@prandla
Copy link
Member

prandla commented Feb 18, 2026

oh, it seems like there might have been a bit of a miscommunication: if we're keeping Participation.contest_id then i don't see any reason not to also keep Contest.participations as a relationship. i reverted that part of your commit; please let me know if you actually had a reason for doing it like that.

also i changed Group.contest_id to be non-null, for i hope obvious reasons. and i fixed a few type hints.

i just realized i can't push changes into this PR since you made it from an organization repo, not a personal one. (github is very good software......) I've pushed my changes into the pr-groups branch of https://github.com/prandla/cms . you can pull from that branch and push it to the pr's branch, or i can just close this PR and create a new one from my own fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@prandla prandla prandla requested changes

+1 more reviewer

@j-bmn j-bmn j-bmn left review comments

Reviewers whose approvals may not affect merge requirements

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@t-lenz @prandla @j-bmn